Goto

Collaborating Authors

 gpt and hallucination


GPTs and Hallucination

Communications of the ACM

LLMs are created by performing machine learning (ML) on large amounts of data. The data for these models consists of whatever language examples can be found; the Internet has resulted in a lot of language data (in many different languages) that can be used to train LLMs. Radically simplifying, the training results in a set of probabilities that can be used to tell, for any word or string of words, which word or words are the most likely to be associated with those words. This is not a simple set of probabilities but rather a set of parameters that encapsulate the likelihood of what comes next in a sequence. Models are often described by the size of the training set and the number of parameters used to build the probability model.